Exploring Different Dimensions of Attention for Uncertainty Detection

摘要

Neural networks with attention have proven effective for many naturallanguage processing tasks. In this paper, we develop attention mechanisms foruncertainty detection. In particular, we generalize standardly used attentionmechanisms by introducing external attention and sequence-preserving attention.These novel architectures differ from standard approaches in that they useexternal resources to compute attention weights and preserve sequenceinformation. We compare them to other configurations along different dimensionsof attention. Our novel architectures set the new state of the art on aWikipedia benchmark dataset and perform similar to the state-of-the-art modelon a biomedical benchmark which uses a large set of linguistic features.